Part 1

mod1= lm(Price~LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF, data=AmesTrain2)

Best Subsets

all=regsubsets(Price~LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF, data=AmesTrain2, nvmax= 22, nbest=2)

#ShowSubsets(all)

The predictors that provide the lowest Mallow’s Cp together are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.

StepWise

MSE=(summary(mod1)$sigma)^2
none=lm(Price~1,data=AmesTrain2)
step(none,scope=list(upper=mod1),scale=MSE)
## Start:  AIC=2973.27
## Price ~ 1
## 
##                   Df Sum of Sq     RSS     Cp
## + GroundSF         1   1688573 1514775 1092.8
## + GarageCars       1   1469716 1733632 1336.7
## + GarageSF         1   1360709 1842639 1458.3
## + FirstSF          1   1281176 1922172 1546.9
## + BasementSF       1   1269388 1933960 1560.1
## + YearBuilt        1   1186095 2017254 1652.9
## + FullBath         1   1057617 2145731 1796.2
## + YearRemodel      1   1015342 2188006 1843.3
## + TotalRooms       1    824832 2378516 2055.7
## + Fireplaces       1    715821 2487528 2177.2
## + OpenPorchSF      1    498984 2704364 2419.0
## + BasementFinSF    1    409618 2793730 2518.6
## + HalfBath         1    368973 2834375 2563.9
## + SecondSF         1    294460 2908888 2647.0
## + LotArea          1    254497 2948851 2691.5
## + Bedroom          1    185248 3018100 2768.7
## + WoodDeckSF       1    164052 3039296 2792.4
## + BasementFBath    1    143206 3060142 2815.6
## + EnclosedPorchSF  1    120421 3082927 2841.0
## + LotFrontage      1     80830 3122518 2885.2
## + ScreenPorchSF    1     76363 3126985 2890.1
## <none>                         3203348 2973.3
## + BasementHBath    1      1675 3201674 2973.4
## 
## Step:  AIC=1092.76
## Price ~ GroundSF
## 
##                   Df Sum of Sq     RSS      Cp
## + YearBuilt        1    576252  938523  452.32
## + BasementSF       1    490057 1024718  548.41
## + GarageCars       1    392203 1122572  657.51
## + YearRemodel      1    384586 1130189  666.00
## + GarageSF         1    362417 1152358  690.71
## + FirstSF          1    350224 1164551  704.31
## + SecondSF         1    320201 1194574  737.78
## + BasementFinSF    1    235650 1279125  832.04
## + Bedroom          1    158563 1356212  917.98
## + BasementFBath    1    119542 1395233  961.48
## + Fireplaces       1     88525 1426250  996.06
## + EnclosedPorchSF  1     85220 1429555  999.75
## + FullBath         1     61884 1452891 1025.76
## + TotalRooms       1     57911 1456864 1030.19
## + LotArea          1     55466 1459309 1032.92
## + OpenPorchSF      1     42173 1472602 1047.74
## + LotFrontage      1     27156 1487619 1064.48
## + WoodDeckSF       1     18801 1495974 1073.79
## + ScreenPorchSF    1     17388 1497387 1075.37
## <none>                         1514775 1092.76
## + HalfBath         1       358 1514417 1094.36
## + BasementHBath    1        76 1514699 1094.67
## - GroundSF         1   1688573 3203348 2973.27
## 
## Step:  AIC=452.32
## Price ~ GroundSF + YearBuilt
## 
##                   Df Sum of Sq     RSS      Cp
## + BasementSF       1    196459  742064  235.29
## + FirstSF          1    168009  770514  267.01
## + SecondSF         1    159185  779338  276.85
## + BasementFinSF    1    115277  823246  325.80
## + GarageSF         1     94124  844399  349.38
## + LotArea          1     73208  865315  372.70
## + GarageCars       1     72790  865732  373.17
## + Bedroom          1     62989  875534  384.09
## + BasementFBath    1     57571  880952  390.13
## + Fireplaces       1     53422  885100  394.76
## + YearRemodel      1     51481  887042  396.92
## + LotFrontage      1     35938  902585  414.25
## + HalfBath         1     32034  906489  418.60
## + ScreenPorchSF    1     26084  912439  425.24
## + OpenPorchSF      1     11941  926582  441.01
## + TotalRooms       1      8387  930135  444.97
## + WoodDeckSF       1      4065  934458  449.79
## + FullBath         1      1873  936649  452.23
## <none>                          938523  452.32
## + EnclosedPorchSF  1      1571  936952  452.57
## + BasementHBath    1        16  938506  454.30
## - YearBuilt        1    576252 1514775 1092.76
## - GroundSF         1   1078731 2017254 1652.95
## 
## Step:  AIC=235.29
## Price ~ GroundSF + YearBuilt + BasementSF
## 
##                   Df Sum of Sq     RSS      Cp
## + YearRemodel      1     60923  681140  169.37
## + GarageSF         1     54583  687481  176.44
## + BasementFinSF    1     42909  699155  189.46
## + Bedroom          1     42382  699681  190.04
## + GarageCars       1     41002  701062  191.58
## + LotArea          1     33506  708558  199.94
## + Fireplaces       1     32163  709901  201.44
## + BasementFBath    1     21030  721034  213.85
## + LotFrontage      1     11533  730531  224.44
## + ScreenPorchSF    1      8187  733877  228.17
## + FirstSF          1      6690  735374  229.84
## + TotalRooms       1      4499  737564  232.28
## + SecondSF         1      4091  737972  232.73
## <none>                          742064  235.29
## + OpenPorchSF      1      1683  740380  235.42
## + WoodDeckSF       1      1613  740451  235.50
## + EnclosedPorchSF  1       821  741243  236.38
## + BasementHBath    1       519  741545  236.72
## + FullBath         1       472  741592  236.77
## + HalfBath         1        59  742005  237.23
## - BasementSF       1    196459  938523  452.32
## - YearBuilt        1    282654 1024718  548.41
## - GroundSF         1    759473 1501537 1080.00
## 
## Step:  AIC=169.37
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel
## 
##                   Df Sum of Sq     RSS     Cp
## + BasementFinSF    1     49265  631875 116.45
## + GarageSF         1     44306  636835 121.98
## + Fireplaces       1     39930  641210 126.86
## + LotArea          1     36176  644964 131.04
## + GarageCars       1     31199  649942 136.59
## + Bedroom          1     26888  654253 141.40
## + BasementFBath    1     19558  661582 149.57
## + LotFrontage      1     11393  669748 158.67
## + ScreenPorchSF    1     11132  670008 158.96
## + FullBath         1      6079  675061 164.60
## + FirstSF          1      5555  675585 165.18
## + TotalRooms       1      3520  677620 167.45
## + SecondSF         1      3245  677895 167.76
## <none>                          681140 169.37
## + EnclosedPorchSF  1      1499  679642 169.70
## + HalfBath         1       716  680425 170.58
## + WoodDeckSF       1       511  680630 170.80
## + OpenPorchSF      1       134  681007 171.22
## + BasementHBath    1        38  681103 171.33
## - YearRemodel      1     60923  742064 235.29
## - YearBuilt        1     94413  775554 272.63
## - BasementSF       1    205902  887042 396.92
## - GroundSF         1    641846 1322987 882.94
## 
## Step:  AIC=116.45
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF
## 
##                   Df Sum of Sq     RSS      Cp
## + GarageSF         1     45612  586263  67.599
## + GarageCars       1     34220  597655  80.299
## + LotArea          1     29908  601967  85.107
## + Fireplaces       1     24808  607067  90.793
## + Bedroom          1     17222  614653  99.249
## + LotFrontage      1     15008  616867 101.718
## + ScreenPorchSF    1      7190  624685 110.434
## + EnclosedPorchSF  1      3332  628543 114.735
## + FirstSF          1      2831  629044 115.293
## + FullBath         1      1908  629967 116.322
## <none>                          631875 116.450
## + SecondSF         1      1475  630400 116.805
## + TotalRooms       1       906  630969 117.439
## + HalfBath         1       810  631065 117.546
## + BasementHBath    1       559  631316 117.826
## + OpenPorchSF      1       324  631551 118.089
## + WoodDeckSF       1        40  631835 118.405
## + BasementFBath    1         3  631872 118.447
## - BasementFinSF    1     49265  681140 169.373
## - YearRemodel      1     67280  699155 189.457
## - YearBuilt        1     81791  713666 205.635
## - BasementSF       1    128293  760168 257.478
## - GroundSF         1    648516 1280391 837.451
## 
## Step:  AIC=67.6
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF
## 
##                   Df Sum of Sq     RSS      Cp
## + LotArea          1     28710  557554  37.592
## + Fireplaces       1     20760  565503  46.455
## + LotFrontage      1      8565  577698  60.050
## + Bedroom          1      8070  578193  60.602
## + ScreenPorchSF    1      5560  580704  63.401
## + HalfBath         1      2578  583685  66.725
## + EnclosedPorchSF  1      2154  584109  67.197
## <none>                          586263  67.599
## + FullBath         1      1336  584927  68.110
## + BasementHBath    1       391  585872  69.163
## + FirstSF          1       373  585890  69.183
## + GarageCars       1       284  585980  69.283
## + TotalRooms       1       218  586045  69.356
## + WoodDeckSF       1       126  586138  69.459
## + BasementFBath    1       126  586138  69.459
## + OpenPorchSF      1       122  586142  69.463
## + SecondSF         1       102  586161  69.485
## - GarageSF         1     45612  631875 116.450
## - YearBuilt        1     49564  635827 120.856
## - BasementFinSF    1     50572  636835 121.979
## - YearRemodel      1     56391  642655 128.467
## - BasementSF       1     99767  686030 176.824
## - GroundSF         1    455152 1041415 573.028
## 
## Step:  AIC=37.59
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea
## 
##                   Df Sum of Sq    RSS      Cp
## + Fireplaces       1     16178 541376  21.556
## + Bedroom          1      8660 548893  29.937
## + LotFrontage      1      6757 550797  32.059
## + ScreenPorchSF    1      5824 551730  33.099
## + HalfBath         1      3170 554384  36.058
## + EnclosedPorchSF  1      3072 554482  36.167
## <none>                         557554  37.592
## + FullBath         1      1675 555879  37.724
## + BasementHBath    1      1408 556145  38.022
## + OpenPorchSF      1       472 557082  39.066
## + GarageCars       1       196 557358  39.373
## + TotalRooms       1       150 557404  39.425
## + FirstSF          1        83 557471  39.500
## + WoodDeckSF       1        35 557519  39.553
## + BasementFBath    1        35 557519  39.553
## + SecondSF         1         4 557550  39.588
## - LotArea          1     28710 586263  67.599
## - BasementFinSF    1     44314 601868  84.996
## - GarageSF         1     44413 601967  85.107
## - YearBuilt        1     56546 614100  98.633
## - YearRemodel      1     58424 615978 100.727
## - BasementSF       1     80372 637926 125.195
## - GroundSF         1    414038 971592 497.185
## 
## Step:  AIC=21.56
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces
## 
##                   Df Sum of Sq    RSS      Cp
## + LotFrontage      1      6812 534564  15.961
## + Bedroom          1      6740 534636  16.042
## + ScreenPorchSF    1      3825 537550  19.291
## + EnclosedPorchSF  1      2894 538482  20.329
## + HalfBath         1      2453 538923  20.821
## <none>                         541376  21.556
## + BasementHBath    1      1485 539890  21.900
## + FullBath         1       834 540541  22.626
## + OpenPorchSF      1       520 540855  22.976
## + WoodDeckSF       1       186 541189  23.348
## + SecondSF         1       159 541217  23.379
## + TotalRooms       1        97 541279  23.448
## + FirstSF          1        66 541310  23.482
## + BasementFBath    1        54 541321  23.495
## + GarageCars       1         6 541370  23.549
## - Fireplaces       1     16178 557554  37.592
## - LotArea          1     24128 565503  46.455
## - BasementFinSF    1     32717 574093  56.030
## - GarageSF         1     40938 582314  65.196
## - YearBuilt        1     53463 594839  79.159
## - YearRemodel      1     62705 604081  89.463
## - BasementSF       1     79162 620537 107.810
## - GroundSF         1    315742 857117 371.562
## 
## Step:  AIC=15.96
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage
## 
##                   Df Sum of Sq    RSS       Cp
## + Bedroom          1      7280 527284   9.8451
## + ScreenPorchSF    1      4171 530393  13.3117
## + HalfBath         1      2825 531738  14.8116
## + EnclosedPorchSF  1      2246 532318  15.4578
## <none>                         534564  15.9615
## + FullBath         1       994 533570  16.8533
## + BasementHBath    1       764 533800  17.1099
## + OpenPorchSF      1       407 534157  17.5075
## + SecondSF         1       336 534228  17.5873
## + TotalRooms       1       268 534296  17.6630
## + FirstSF          1       185 534379  17.7550
## + WoodDeckSF       1        31 534533  17.9273
## + GarageCars       1        11 534553  17.9498
## + BasementFBath    1         9 534555  17.9512
## - LotFrontage      1      6812 541376  21.5557
## - Fireplaces       1     16233 550797  32.0592
## - LotArea          1     22476 557040  39.0193
## - BasementFinSF    1     34709 569273  52.6574
## - GarageSF         1     35471 570035  53.5069
## - YearBuilt        1     57080 591643  77.5971
## - YearRemodel      1     63282 597845  84.5115
## - BasementSF       1     69635 604199  91.5945
## - GroundSF         1    317663 852227 368.1097
## 
## Step:  AIC=9.85
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom
## 
##                   Df Sum of Sq    RSS       Cp
## + ScreenPorchSF    1      4063 523220   7.3153
## + HalfBath         1      2361 524922   9.2126
## <none>                         527284   9.8451
## + EnclosedPorchSF  1      1693 525590   9.9573
## + TotalRooms       1       525 526759  11.2598
## + SecondSF         1       463 526820  11.3288
## + FullBath         1       438 526846  11.3571
## + OpenPorchSF      1       322 526961  11.4857
## + FirstSF          1       314 526970  11.4952
## + BasementHBath    1       203 527081  11.6193
## + WoodDeckSF       1        52 527231  11.7871
## + BasementFBath    1        26 527257  11.8157
## + GarageCars       1         0 527284  11.8451
## - Bedroom          1      7280 534564  15.9615
## - LotFrontage      1      7352 534636  16.0418
## - Fireplaces       1     14241 541525  23.7222
## - LotArea          1     23125 550408  33.6258
## - GarageSF         1     27990 555274  39.0500
## - BasementFinSF    1     29808 557091  41.0763
## - YearRemodel      1     53823 581106  67.8499
## - YearBuilt        1     59009 586292  73.6313
## - BasementSF       1     68154 595438  83.8272
## - GroundSF         1    241622 768905 277.2185
## 
## Step:  AIC=7.32
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## + EnclosedPorchSF  1      2268 520953   6.7871
## + HalfBath         1      1940 521281   7.1528
## <none>                         523220   7.3153
## + TotalRooms       1       666 522554   8.5726
## + SecondSF         1       504 522717   8.7535
## + OpenPorchSF      1       397 522823   8.8727
## + FirstSF          1       361 522860   8.9132
## + FullBath         1       257 522963   9.0283
## + BasementHBath    1       250 522971   9.0368
## + BasementFBath    1        18 523202   9.2953
## + GarageCars       1        13 523208   9.3013
## + WoodDeckSF       1         2 523219   9.3136
## - ScreenPorchSF    1      4063 527284   9.8451
## - Bedroom          1      7173 530393  13.3117
## - LotFrontage      1      7702 530922  13.9015
## - Fireplaces       1     12356 535576  19.0901
## - LotArea          1     23509 546730  31.5247
## - GarageSF         1     27037 550257  35.4573
## - BasementFinSF    1     28213 551434  36.7691
## - YearRemodel      1     55235 578456  66.8948
## - YearBuilt        1     61090 584310  73.4218
## - BasementSF       1     63477 586697  76.0824
## - GroundSF         1    240344 763565 273.2643
## 
## Step:  AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF + EnclosedPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## + HalfBath         1      1795 519157   6.7855
## <none>                         520953   6.7871
## - EnclosedPorchSF  1      2268 523220   7.3153
## + TotalRooms       1       606 520346   8.1112
## + OpenPorchSF      1       376 520577   8.3684
## + FullBath         1       281 520672   8.4737
## + SecondSF         1       244 520709   8.5155
## + BasementHBath    1       232 520721   8.5283
## + FirstSF          1       135 520817   8.6363
## + BasementFBath    1        32 520921   8.7514
## + GarageCars       1        17 520936   8.7681
## + WoodDeckSF       1         1 520952   8.7865
## - ScreenPorchSF    1      4637 525590   9.9573
## - Bedroom          1      6535 527488  12.0727
## - LotFrontage      1      7004 527957  12.5953
## - Fireplaces       1     12164 533116  18.3477
## - LotArea          1     24298 545251  31.8763
## - GarageSF         1     26545 547497  34.3805
## - BasementFinSF    1     29306 550258  37.4586
## - YearRemodel      1     56510 577463  67.7880
## - YearBuilt        1     62029 582982  73.9404
## - BasementSF       1     62342 583294  74.2891
## - GroundSF         1    233848 754801 265.4941
## 
## Step:  AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF + EnclosedPorchSF + HalfBath
## 
##                   Df Sum of Sq    RSS       Cp
## <none>                         519157   6.7855
## - HalfBath         1      1795 520953   6.7871
## - EnclosedPorchSF  1      2123 521281   7.1528
## + TotalRooms       1       705 518452   7.9996
## + OpenPorchSF      1       205 518952   8.5572
## + BasementHBath    1       171 518986   8.5948
## + BasementFBath    1        41 519117   8.7403
## + GarageCars       1        16 519142   8.7682
## + FirstSF          1        15 519142   8.7686
## + FullBath         1        13 519145   8.7716
## + WoodDeckSF       1         0 519157   8.7855
## + SecondSF         1         0 519157   8.7855
## - ScreenPorchSF    1      4181 523339   9.4469
## - Bedroom          1      6174 525332  11.6691
## - LotFrontage      1      7291 526448  12.9136
## - Fireplaces       1     11778 530935  17.9161
## - LotArea          1     24678 543835  32.2977
## - GarageSF         1     27771 546929  35.7465
## - BasementFinSF    1     29712 548870  37.9104
## - YearBuilt        1     47897 567054  58.1832
## - YearRemodel      1     57616 576774  69.0193
## - BasementSF       1     61159 580317  72.9695
## - GroundSF         1    174283 693440 199.0855
## 
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
##     BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
##     Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
## 
## Coefficients:
##     (Intercept)         GroundSF        YearBuilt       BasementSF  
##      -2.126e+03        6.635e-02        4.613e-01        3.668e-02  
##     YearRemodel    BasementFinSF         GarageSF          LotArea  
##       6.174e-01        1.951e-02        4.164e-02        5.923e-04  
##      Fireplaces      LotFrontage          Bedroom    ScreenPorchSF  
##       8.402e+00        1.149e-01       -5.542e+00        4.200e-02  
## EnclosedPorchSF         HalfBath  
##       3.737e-02        4.462e+00

The best predictors together according to stepwise regression are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.

Backward Elimination

MSE=(summary(mod1)$sigma)^2
step(mod1,scale=MSE)
## Start:  AIC=23
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + 
##     BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF + 
##     EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - GarageCars       1         1 517557 21.001
## - WoodDeckSF       1         9 517564 21.010
## - FullBath         1        84 517639 21.093
## - BasementFBath    1       167 517723 21.186
## - BasementHBath    1       201 517756 21.224
## - SecondSF         1       279 517835 21.311
## - FirstSF          1       283 517839 21.316
## - OpenPorchSF      1       357 517913 21.398
## - TotalRooms       1       704 518260 21.785
## - HalfBath         1      1022 518577 22.139
## - GroundSF         1      1296 518852 22.445
## <none>                         517556 23.000
## - EnclosedPorchSF  1      2123 519679 23.367
## - ScreenPorchSF    1      4301 521857 25.795
## - Bedroom          1      5958 523514 27.642
## - LotFrontage      1      6472 524028 28.216
## - GarageSF         1      8606 526162 30.595
## - Fireplaces       1     10574 528130 32.789
## - BasementFinSF    1     19647 537203 42.904
## - LotArea          1     25105 542661 48.989
## - BasementSF       1     25690 543246 49.641
## - YearBuilt        1     42314 559870 68.174
## - YearRemodel      1     52874 570430 79.947
## 
## Step:  AIC=21
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + 
##     BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF + 
##     ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - WoodDeckSF       1         9 517566 19.011
## - FullBath         1        83 517640 19.093
## - BasementFBath    1       168 517725 19.189
## - BasementHBath    1       200 517757 19.225
## - SecondSF         1       280 517837 19.313
## - FirstSF          1       283 517840 19.317
## - OpenPorchSF      1       356 517913 19.398
## - TotalRooms       1       723 518280 19.807
## - HalfBath         1      1023 518580 20.142
## - GroundSF         1      1296 518853 20.446
## <none>                         517557 21.001
## - EnclosedPorchSF  1      2122 519679 21.367
## - ScreenPorchSF    1      4303 521860 23.799
## - Bedroom          1      6038 523595 25.733
## - LotFrontage      1      6471 524028 26.216
## - Fireplaces       1     10696 528253 30.926
## - BasementFinSF    1     19649 537206 40.907
## - LotArea          1     25111 542668 46.997
## - GarageSF         1     25443 543000 47.367
## - BasementSF       1     25848 543405 47.818
## - YearBuilt        1     44031 561588 68.089
## - YearRemodel      1     52930 570487 78.011
## 
## Step:  AIC=19.01
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + 
##     BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - FullBath         1        82 517648 17.103
## - BasementFBath    1       164 517730 17.194
## - BasementHBath    1       195 517761 17.229
## - SecondSF         1       278 517844 17.322
## - FirstSF          1       281 517847 17.325
## - OpenPorchSF      1       349 517915 17.400
## - TotalRooms       1       723 518289 17.817
## - HalfBath         1      1031 518597 18.161
## - GroundSF         1      1303 518869 18.464
## <none>                         517566 19.011
## - EnclosedPorchSF  1      2114 519680 19.368
## - ScreenPorchSF    1      4309 521875 21.815
## - Bedroom          1      6056 523622 23.763
## - LotFrontage      1      6478 524044 24.233
## - Fireplaces       1     10792 528357 29.042
## - BasementFinSF    1     19709 537275 38.984
## - LotArea          1     25127 542693 45.025
## - GarageSF         1     25543 543109 45.489
## - BasementSF       1     25941 543507 45.932
## - YearBuilt        1     44075 561641 66.149
## - YearRemodel      1     53310 570876 76.445
## 
## Step:  AIC=17.1
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + 
##     BasementHBath + HalfBath + Bedroom + TotalRooms + Fireplaces + 
##     GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - BasementFBath    1       133 517781 15.251
## - BasementHBath    1       201 517849 15.327
## - SecondSF         1       269 517917 15.402
## - FirstSF          1       275 517923 15.409
## - OpenPorchSF      1       333 517981 15.474
## - TotalRooms       1       671 518319 15.851
## - GroundSF         1      1283 518931 16.533
## - HalfBath         1      1352 519000 16.610
## <none>                         517648 17.103
## - EnclosedPorchSF  1      2095 519743 17.439
## - ScreenPorchSF    1      4379 522026 19.984
## - Bedroom          1      6105 523753 21.909
## - LotFrontage      1      6456 524104 22.300
## - Fireplaces       1     10978 528626 27.342
## - BasementFinSF    1     19699 537346 37.064
## - LotArea          1     25062 542710 43.043
## - GarageSF         1     25793 543440 43.858
## - BasementSF       1     26108 543756 44.210
## - YearBuilt        1     48072 565720 68.696
## - YearRemodel      1     55056 572704 76.482
## 
## Step:  AIC=15.25
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + BasementHBath + 
##     HalfBath + Bedroom + TotalRooms + Fireplaces + GarageSF + 
##     OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - BasementHBath    1       133 517914 13.399
## - SecondSF         1       258 518039 13.539
## - FirstSF          1       267 518048 13.549
## - OpenPorchSF      1       303 518084 13.589
## - TotalRooms       1       676 518457 14.005
## - GroundSF         1      1310 519092 14.712
## - HalfBath         1      1368 519149 14.777
## <none>                         517781 15.251
## - EnclosedPorchSF  1      2072 519853 15.561
## - ScreenPorchSF    1      4382 522163 18.136
## - Bedroom          1      6039 523821 19.984
## - LotFrontage      1      6428 524209 20.418
## - Fireplaces       1     11006 528787 25.521
## - LotArea          1     24929 542710 41.044
## - BasementSF       1     26038 543819 42.280
## - GarageSF         1     26269 544050 42.538
## - BasementFinSF    1     30139 547920 46.852
## - YearBuilt        1     48039 565820 66.808
## - YearRemodel      1     54922 572704 74.482
## 
## Step:  AIC=13.4
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + SecondSF + GroundSF + HalfBath + Bedroom + 
##     TotalRooms + Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + 
##     ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## - SecondSF         1       251 518165 11.679
## - FirstSF          1       258 518172 11.687
## - OpenPorchSF      1       291 518205 11.723
## - TotalRooms       1       739 518653 12.223
## - GroundSF         1      1333 519247 12.885
## - HalfBath         1      1414 519328 12.976
## <none>                         517914 13.399
## - EnclosedPorchSF  1      2076 519990 13.714
## - ScreenPorchSF    1      4348 522262 16.246
## - Bedroom          1      6590 524503 18.746
## - LotFrontage      1      6821 524734 19.003
## - Fireplaces       1     10963 528877 23.621
## - LotArea          1     24821 542735 39.071
## - BasementSF       1     26110 544024 40.508
## - GarageSF         1     26205 544119 40.614
## - BasementFinSF    1     30007 547921 44.853
## - YearBuilt        1     47936 565850 64.841
## - YearRemodel      1     55160 573074 72.895
## 
## Step:  AIC=11.68
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + FirstSF + GroundSF + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## - FirstSF          1         8 518173   9.6877
## - OpenPorchSF      1       272 518437   9.9820
## - TotalRooms       1       779 518944  10.5480
## - HalfBath         1      1551 519716  11.4087
## <none>                         518165  11.6791
## - EnclosedPorchSF  1      2031 520196  11.9438
## - ScreenPorchSF    1      4376 522541  14.5575
## - Bedroom          1      6759 524924  17.2142
## - LotFrontage      1      6785 524950  17.2431
## - Fireplaces       1     11261 529426  22.2335
## - LotArea          1     25109 543274  37.6717
## - BasementSF       1     25923 544088  38.5794
## - GarageSF         1     27118 545283  39.9121
## - BasementFinSF    1     30176 548341  43.3214
## - YearBuilt        1     48699 566864  63.9714
## - YearRemodel      1     55175 573340  71.1913
## - GroundSF         1     95325 613490 115.9529
## 
## Step:  AIC=9.69
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + GroundSF + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## - OpenPorchSF      1       280 518452   7.9996
## - TotalRooms       1       780 518952   8.5572
## - HalfBath         1      1700 519872   9.5828
## <none>                         518173   9.6877
## - EnclosedPorchSF  1      2042 520215   9.9646
## - ScreenPorchSF    1      4395 522568  12.5880
## - Bedroom          1      6833 525006  15.3055
## - LotFrontage      1      6848 525021  15.3227
## - Fireplaces       1     11640 529813  20.6648
## - LotArea          1     25159 543332  35.7364
## - GarageSF         1     27475 545647  38.3181
## - BasementFinSF    1     30283 548455  41.4487
## - YearBuilt        1     48793 566966  62.0851
## - YearRemodel      1     55171 573343  69.1950
## - BasementSF       1     57499 575672  71.7913
## - GroundSF         1    103586 621759 123.1714
## 
## Step:  AIC=8
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + GroundSF + HalfBath + Bedroom + TotalRooms + 
##     Fireplaces + GarageSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## - TotalRooms       1       705 519157   6.7855
## <none>                         518452   7.9996
## - HalfBath         1      1894 520346   8.1112
## - EnclosedPorchSF  1      2057 520509   8.2929
## - ScreenPorchSF    1      4309 522762  10.8039
## - Bedroom          1      6778 525231  13.5566
## - LotFrontage      1      6977 525430  13.7784
## - Fireplaces       1     11606 530058  18.9386
## - LotArea          1     24926 543378  33.7882
## - GarageSF         1     27731 546184  36.9161
## - BasementFinSF    1     30172 548625  39.6374
## - YearBuilt        1     48517 566969  60.0889
## - YearRemodel      1     56895 575347  69.4288
## - BasementSF       1     61263 579715  74.2991
## - GroundSF         1    108474 626927 126.9328
## 
## Step:  AIC=6.79
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + 
##     BasementSF + GroundSF + HalfBath + Bedroom + Fireplaces + 
##     GarageSF + EnclosedPorchSF + ScreenPorchSF
## 
##                   Df Sum of Sq    RSS       Cp
## <none>                         519157   6.7855
## - HalfBath         1      1795 520953   6.7871
## - EnclosedPorchSF  1      2123 521281   7.1528
## - ScreenPorchSF    1      4181 523339   9.4469
## - Bedroom          1      6174 525332  11.6691
## - LotFrontage      1      7291 526448  12.9136
## - Fireplaces       1     11778 530935  17.9161
## - LotArea          1     24678 543835  32.2977
## - GarageSF         1     27771 546929  35.7465
## - BasementFinSF    1     29712 548870  37.9104
## - YearBuilt        1     47897 567054  58.1832
## - YearRemodel      1     57616 576774  69.0193
## - BasementSF       1     61159 580317  72.9695
## - GroundSF         1    174283 693440 199.0855
## 
## Call:
## lm(formula = Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + 
##     BasementFinSF + BasementSF + GroundSF + HalfBath + Bedroom + 
##     Fireplaces + GarageSF + EnclosedPorchSF + ScreenPorchSF, 
##     data = AmesTrain2)
## 
## Coefficients:
##     (Intercept)      LotFrontage          LotArea        YearBuilt  
##      -2.126e+03        1.149e-01        5.923e-04        4.613e-01  
##     YearRemodel    BasementFinSF       BasementSF         GroundSF  
##       6.174e-01        1.951e-02        3.668e-02        6.635e-02  
##        HalfBath          Bedroom       Fireplaces         GarageSF  
##       4.462e+00       -5.542e+00        8.402e+00        4.164e-02  
## EnclosedPorchSF    ScreenPorchSF  
##       3.737e-02        4.200e-02

The best predictors together according to Backward Elimination are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.

Forward Selection

none=lm(Price~1,data=AmesTrain2)
step(none,scope=list(upper=mod1),scale=MSE, direction="forward")
## Start:  AIC=2973.27
## Price ~ 1
## 
##                   Df Sum of Sq     RSS     Cp
## + GroundSF         1   1688573 1514775 1092.8
## + GarageCars       1   1469716 1733632 1336.7
## + GarageSF         1   1360709 1842639 1458.3
## + FirstSF          1   1281176 1922172 1546.9
## + BasementSF       1   1269388 1933960 1560.1
## + YearBuilt        1   1186095 2017254 1652.9
## + FullBath         1   1057617 2145731 1796.2
## + YearRemodel      1   1015342 2188006 1843.3
## + TotalRooms       1    824832 2378516 2055.7
## + Fireplaces       1    715821 2487528 2177.2
## + OpenPorchSF      1    498984 2704364 2419.0
## + BasementFinSF    1    409618 2793730 2518.6
## + HalfBath         1    368973 2834375 2563.9
## + SecondSF         1    294460 2908888 2647.0
## + LotArea          1    254497 2948851 2691.5
## + Bedroom          1    185248 3018100 2768.7
## + WoodDeckSF       1    164052 3039296 2792.4
## + BasementFBath    1    143206 3060142 2815.6
## + EnclosedPorchSF  1    120421 3082927 2841.0
## + LotFrontage      1     80830 3122518 2885.2
## + ScreenPorchSF    1     76363 3126985 2890.1
## <none>                         3203348 2973.3
## + BasementHBath    1      1675 3201674 2973.4
## 
## Step:  AIC=1092.76
## Price ~ GroundSF
## 
##                   Df Sum of Sq     RSS      Cp
## + YearBuilt        1    576252  938523  452.32
## + BasementSF       1    490057 1024718  548.41
## + GarageCars       1    392203 1122572  657.51
## + YearRemodel      1    384586 1130189  666.00
## + GarageSF         1    362417 1152358  690.71
## + FirstSF          1    350224 1164551  704.31
## + SecondSF         1    320201 1194574  737.78
## + BasementFinSF    1    235650 1279125  832.04
## + Bedroom          1    158563 1356212  917.98
## + BasementFBath    1    119542 1395233  961.48
## + Fireplaces       1     88525 1426250  996.06
## + EnclosedPorchSF  1     85220 1429555  999.75
## + FullBath         1     61884 1452891 1025.76
## + TotalRooms       1     57911 1456864 1030.19
## + LotArea          1     55466 1459309 1032.92
## + OpenPorchSF      1     42173 1472602 1047.74
## + LotFrontage      1     27156 1487619 1064.48
## + WoodDeckSF       1     18801 1495974 1073.79
## + ScreenPorchSF    1     17388 1497387 1075.37
## <none>                         1514775 1092.76
## + HalfBath         1       358 1514417 1094.36
## + BasementHBath    1        76 1514699 1094.67
## 
## Step:  AIC=452.32
## Price ~ GroundSF + YearBuilt
## 
##                   Df Sum of Sq    RSS     Cp
## + BasementSF       1    196459 742064 235.29
## + FirstSF          1    168009 770514 267.01
## + SecondSF         1    159185 779338 276.85
## + BasementFinSF    1    115277 823246 325.80
## + GarageSF         1     94124 844399 349.38
## + LotArea          1     73208 865315 372.70
## + GarageCars       1     72790 865732 373.17
## + Bedroom          1     62989 875534 384.09
## + BasementFBath    1     57571 880952 390.13
## + Fireplaces       1     53422 885100 394.76
## + YearRemodel      1     51481 887042 396.92
## + LotFrontage      1     35938 902585 414.25
## + HalfBath         1     32034 906489 418.60
## + ScreenPorchSF    1     26084 912439 425.24
## + OpenPorchSF      1     11941 926582 441.01
## + TotalRooms       1      8387 930135 444.97
## + WoodDeckSF       1      4065 934458 449.79
## + FullBath         1      1873 936649 452.23
## <none>                         938523 452.32
## + EnclosedPorchSF  1      1571 936952 452.57
## + BasementHBath    1        16 938506 454.30
## 
## Step:  AIC=235.29
## Price ~ GroundSF + YearBuilt + BasementSF
## 
##                   Df Sum of Sq    RSS     Cp
## + YearRemodel      1     60923 681140 169.37
## + GarageSF         1     54583 687481 176.44
## + BasementFinSF    1     42909 699155 189.46
## + Bedroom          1     42382 699681 190.04
## + GarageCars       1     41002 701062 191.58
## + LotArea          1     33506 708558 199.94
## + Fireplaces       1     32163 709901 201.44
## + BasementFBath    1     21030 721034 213.85
## + LotFrontage      1     11533 730531 224.44
## + ScreenPorchSF    1      8187 733877 228.17
## + FirstSF          1      6690 735374 229.84
## + TotalRooms       1      4499 737564 232.28
## + SecondSF         1      4091 737972 232.73
## <none>                         742064 235.29
## + OpenPorchSF      1      1683 740380 235.42
## + WoodDeckSF       1      1613 740451 235.50
## + EnclosedPorchSF  1       821 741243 236.38
## + BasementHBath    1       519 741545 236.72
## + FullBath         1       472 741592 236.77
## + HalfBath         1        59 742005 237.23
## 
## Step:  AIC=169.37
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel
## 
##                   Df Sum of Sq    RSS     Cp
## + BasementFinSF    1     49265 631875 116.45
## + GarageSF         1     44306 636835 121.98
## + Fireplaces       1     39930 641210 126.86
## + LotArea          1     36176 644964 131.04
## + GarageCars       1     31199 649942 136.59
## + Bedroom          1     26888 654253 141.40
## + BasementFBath    1     19558 661582 149.57
## + LotFrontage      1     11393 669748 158.67
## + ScreenPorchSF    1     11132 670008 158.96
## + FullBath         1      6079 675061 164.60
## + FirstSF          1      5555 675585 165.18
## + TotalRooms       1      3520 677620 167.45
## + SecondSF         1      3245 677895 167.76
## <none>                         681140 169.37
## + EnclosedPorchSF  1      1499 679642 169.70
## + HalfBath         1       716 680425 170.58
## + WoodDeckSF       1       511 680630 170.80
## + OpenPorchSF      1       134 681007 171.22
## + BasementHBath    1        38 681103 171.33
## 
## Step:  AIC=116.45
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF
## 
##                   Df Sum of Sq    RSS      Cp
## + GarageSF         1     45612 586263  67.599
## + GarageCars       1     34220 597655  80.299
## + LotArea          1     29908 601967  85.107
## + Fireplaces       1     24808 607067  90.793
## + Bedroom          1     17222 614653  99.249
## + LotFrontage      1     15008 616867 101.718
## + ScreenPorchSF    1      7190 624685 110.434
## + EnclosedPorchSF  1      3332 628543 114.735
## + FirstSF          1      2831 629044 115.293
## + FullBath         1      1908 629967 116.322
## <none>                         631875 116.450
## + SecondSF         1      1475 630400 116.805
## + TotalRooms       1       906 630969 117.439
## + HalfBath         1       810 631065 117.546
## + BasementHBath    1       559 631316 117.826
## + OpenPorchSF      1       324 631551 118.089
## + WoodDeckSF       1        40 631835 118.405
## + BasementFBath    1         3 631872 118.447
## 
## Step:  AIC=67.6
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF
## 
##                   Df Sum of Sq    RSS     Cp
## + LotArea          1   28709.5 557554 37.592
## + Fireplaces       1   20760.0 565503 46.455
## + LotFrontage      1    8564.8 577698 60.050
## + Bedroom          1    8070.2 578193 60.602
## + ScreenPorchSF    1    5559.7 580704 63.401
## + HalfBath         1    2577.8 583685 66.725
## + EnclosedPorchSF  1    2154.2 584109 67.197
## <none>                         586263 67.599
## + FullBath         1    1335.8 584927 68.110
## + BasementHBath    1     391.2 585872 69.163
## + FirstSF          1     373.3 585890 69.183
## + GarageCars       1     283.8 585980 69.283
## + TotalRooms       1     217.9 586045 69.356
## + WoodDeckSF       1     125.8 586138 69.459
## + BasementFBath    1     125.6 586138 69.459
## + OpenPorchSF      1     121.6 586142 69.463
## + SecondSF         1     102.5 586161 69.485
## 
## Step:  AIC=37.59
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea
## 
##                   Df Sum of Sq    RSS     Cp
## + Fireplaces       1   16178.1 541376 21.556
## + Bedroom          1    8660.4 548893 29.937
## + LotFrontage      1    6756.8 550797 32.059
## + ScreenPorchSF    1    5824.1 551730 33.099
## + HalfBath         1    3170.0 554384 36.058
## + EnclosedPorchSF  1    3072.2 554482 36.167
## <none>                         557554 37.592
## + FullBath         1    1675.2 555879 37.724
## + BasementHBath    1    1408.3 556145 38.022
## + OpenPorchSF      1     471.8 557082 39.066
## + GarageCars       1     196.2 557358 39.373
## + TotalRooms       1     149.8 557404 39.425
## + FirstSF          1      82.5 557471 39.500
## + WoodDeckSF       1      34.8 557519 39.553
## + BasementFBath    1      34.6 557519 39.553
## + SecondSF         1       3.8 557550 39.588
## 
## Step:  AIC=21.56
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces
## 
##                   Df Sum of Sq    RSS     Cp
## + LotFrontage      1    6811.8 534564 15.961
## + Bedroom          1    6739.8 534636 16.042
## + ScreenPorchSF    1    3825.2 537550 19.291
## + EnclosedPorchSF  1    2893.9 538482 20.329
## + HalfBath         1    2452.7 538923 20.821
## <none>                         541376 21.556
## + BasementHBath    1    1485.2 539890 21.900
## + FullBath         1     834.2 540541 22.626
## + OpenPorchSF      1     520.2 540855 22.976
## + WoodDeckSF       1     186.5 541189 23.348
## + SecondSF         1     158.9 541217 23.379
## + TotalRooms       1      96.5 541279 23.448
## + FirstSF          1      66.0 541310 23.482
## + BasementFBath    1      54.4 541321 23.495
## + GarageCars       1       5.7 541370 23.549
## 
## Step:  AIC=15.96
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage
## 
##                   Df Sum of Sq    RSS      Cp
## + Bedroom          1    7280.2 527284  9.8451
## + ScreenPorchSF    1    4170.7 530393 13.3117
## + HalfBath         1    2825.4 531738 14.8116
## + EnclosedPorchSF  1    2245.8 532318 15.4578
## <none>                         534564 15.9615
## + FullBath         1     994.0 533570 16.8533
## + BasementHBath    1     763.9 533800 17.1099
## + OpenPorchSF      1     407.3 534157 17.5075
## + SecondSF         1     335.7 534228 17.5873
## + TotalRooms       1     267.8 534296 17.6630
## + FirstSF          1     185.2 534379 17.7550
## + WoodDeckSF       1      30.7 534533 17.9273
## + GarageCars       1      10.5 534553 17.9498
## + BasementFBath    1       9.3 534555 17.9512
## 
## Step:  AIC=9.85
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom
## 
##                   Df Sum of Sq    RSS      Cp
## + ScreenPorchSF    1    4063.1 523220  7.3153
## + HalfBath         1    2361.3 524922  9.2126
## <none>                         527284  9.8451
## + EnclosedPorchSF  1    1693.4 525590  9.9573
## + TotalRooms       1     525.0 526759 11.2598
## + SecondSF         1     463.1 526820 11.3288
## + FullBath         1     437.7 526846 11.3571
## + OpenPorchSF      1     322.4 526961 11.4857
## + FirstSF          1     313.9 526970 11.4952
## + BasementHBath    1     202.5 527081 11.6193
## + WoodDeckSF       1      52.1 527231 11.7871
## + BasementFBath    1      26.3 527257 11.8157
## + GarageCars       1       0.0 527284 11.8451
## 
## Step:  AIC=7.32
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF
## 
##                   Df Sum of Sq    RSS     Cp
## + EnclosedPorchSF  1   2267.69 520953 6.7871
## + HalfBath         1   1939.71 521281 7.1528
## <none>                         523220 7.3153
## + TotalRooms       1    666.18 522554 8.5726
## + SecondSF         1    503.87 522717 8.7535
## + OpenPorchSF      1    397.00 522823 8.8727
## + FirstSF          1    360.63 522860 8.9132
## + FullBath         1    257.43 522963 9.0283
## + BasementHBath    1    249.76 522971 9.0368
## + BasementFBath    1     17.94 523202 9.2953
## + GarageCars       1     12.58 523208 9.3013
## + WoodDeckSF       1      1.52 523219 9.3136
## 
## Step:  AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF + EnclosedPorchSF
## 
##                 Df Sum of Sq    RSS     Cp
## + HalfBath       1   1795.41 519157 6.7855
## <none>                       520953 6.7871
## + TotalRooms     1    606.31 520346 8.1112
## + OpenPorchSF    1    375.57 520577 8.3684
## + FullBath       1    281.18 520672 8.4737
## + SecondSF       1    243.62 520709 8.5155
## + BasementHBath  1    232.20 520721 8.5283
## + FirstSF        1    135.33 520817 8.6363
## + BasementFBath  1     32.02 520921 8.7514
## + GarageCars     1     17.04 520936 8.7681
## + WoodDeckSF     1      0.57 520952 8.7865
## 
## Step:  AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF + 
##     GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom + 
##     ScreenPorchSF + EnclosedPorchSF + HalfBath
## 
##                 Df Sum of Sq    RSS     Cp
## <none>                       519157 6.7855
## + TotalRooms     1    704.95 518452 7.9996
## + OpenPorchSF    1    204.81 518952 8.5572
## + BasementHBath  1    171.06 518986 8.5948
## + BasementFBath  1     40.59 519117 8.7403
## + GarageCars     1     15.56 519142 8.7682
## + FirstSF        1     15.13 519142 8.7686
## + FullBath       1     12.50 519145 8.7716
## + WoodDeckSF     1      0.04 519157 8.7855
## + SecondSF       1      0.01 519157 8.7855
## 
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
##     BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
##     Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
## 
## Coefficients:
##     (Intercept)         GroundSF        YearBuilt       BasementSF  
##      -2.126e+03        6.635e-02        4.613e-01        3.668e-02  
##     YearRemodel    BasementFinSF         GarageSF          LotArea  
##       6.174e-01        1.951e-02        4.164e-02        5.923e-04  
##      Fireplaces      LotFrontage          Bedroom    ScreenPorchSF  
##       8.402e+00        1.149e-01       -5.542e+00        4.200e-02  
## EnclosedPorchSF         HalfBath  
##       3.737e-02        4.462e+00

The best predictors together according to Forward Selection are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.

After using the methods of Best subsets, stepwise regression, backward elimination, and forward selection, we have determined that the best model to predict Price uses the following predictors: LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.

Every method gave us these predictors as the best predictors to use for a model because these predictors had the lowest Mallow’s Cp. The lower the Mallow’s Cp, the better the compromise between having little error in the model and not having too many predictors. In this case, this combination of predictors gave a Mallow’s Cp of 6.79, which was the lowest possible Mallow’s Cp given the predictors.

Here’s the summary for our model:

mod2= lm(Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
    BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
    Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
summary(mod2)
## 
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
##     BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
##     Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -148.184  -16.828   -1.793   14.028  183.353 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     -2.126e+03  1.465e+02 -14.509  < 2e-16 ***
## GroundSF         6.635e-02  4.731e-03  14.026  < 2e-16 ***
## YearBuilt        4.613e-01  6.273e-02   7.353 6.57e-13 ***
## BasementSF       3.668e-02  4.415e-03   8.309 6.75e-16 ***
## YearRemodel      6.174e-01  7.655e-02   8.064 4.17e-15 ***
## BasementFinSF    1.951e-02  3.368e-03   5.791 1.14e-08 ***
## GarageSF         4.164e-02  7.437e-03   5.599 3.32e-08 ***
## LotArea          5.923e-04  1.122e-04   5.278 1.84e-07 ***
## Fireplaces       8.402e+00  2.304e+00   3.646  0.00029 ***
## LotFrontage      1.149e-01  4.004e-02   2.869  0.00427 ** 
## Bedroom         -5.542e+00  2.099e+00  -2.640  0.00851 ** 
## ScreenPorchSF    4.200e-02  1.933e-02   2.172  0.03022 *  
## EnclosedPorchSF  3.737e-02  2.414e-02   1.548  0.12212    
## HalfBath         4.462e+00  3.135e+00   1.424  0.15510    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 29.76 on 586 degrees of freedom
## Multiple R-squared:  0.8379, Adjusted R-squared:  0.8343 
## F-statistic: 233.1 on 13 and 586 DF,  p-value: < 2.2e-16

The following predictors are not significant at a 5% level in this model: EnclosedPorchSF and HalfBath. However, this does not necessarily mean they are bad predictors, as their higher p-values may be inflated due to multicollinearity. Additionally, their p-values are both still below .2, so while they are not significant at a 5% level, that does not necessarily mean they are bad predictors.

Let’s look into the VIF values:

vif(mod2)
##        GroundSF       YearBuilt      BasementSF     YearRemodel 
##        3.359649        2.588815        1.949760        1.773424 
##   BasementFinSF        GarageSF         LotArea      Fireplaces 
##        1.289299        1.832199        1.126276        1.394249 
##     LotFrontage         Bedroom   ScreenPorchSF EnclosedPorchSF 
##        1.090803        1.787478        1.090471        1.278974 
##        HalfBath 
##        1.700918

VIF is used to detect multicollinearity. A VIF value greater than 5 generally causes concern for substantial multicollinearity, but a value of around 2 or 3 to 5 can still cause concern for moderate multicollinearity. No predictors have a VIF value greater than 5, but there are 2 predictors with VIF values greater than 2.5 that cause concern for moderate multicollinearity. These predictors are GroundSF and YearBuilt. The rest of the predictors have VIF values of between 1 and 2, causing little concern for multicollinearity.

Part 2

First we’ll check for linearity

plot(mod2)

hist(mod2$residuals)

Looking at the residuals vs fitted values plot, the linearity condition does not seem to be fully met. There appears to be curvature of the residuals, as it dips down and then curves up, suggesting that linearity is not met.

The zero mean condition seems to be met somewhat well, as the residuals seem to be generally spread around the zero line in the residuals vs. fitted values plot. The zero line seems to be close to passing through the center of the data, but it doesn’t quite do so perfectly.

The constant variance condition does not appear to be met, as there is definitely greater variance as fitted values reach around 250 and beyond than before they reach 250. The data is definitely more clustered in the middle of the plot, and there is also some curvature.

Looking at the normal q-q plot and the histogram, the independence condition appears to be reasonably met. In the normal q-q plot, the points generally follow the line for the most part, although they trail off a little at both tails. The histogram has a general bell curve and does not have a strong skew, further suggesting that the independence condition is met.

Checking for unusual studentized residuals

which(rstudent(mod2) > 3)
##  62  70 198 202 374 537 572 581 
##  62  70 198 202 374 537 572 581

The criteria we are using to identify “unsual” cases for studentized residuals is if the studentized residual is greater than 3, then it is “unusual.” Based on this criteria, the following indices are unsusual in terms of studentized residuals: 62, 70 198, 202, 374, 537, 572, 581. It is important to note that just because these points are unusual in terms of their studentized residuals doesn’t mean that they are influential.

Checking for influential points

To check to see if these points are influential, we’ll see if the studentized residuals are different or similar to the standardized residuals. If they are similar for a point, the point is not very influential.

rstudent(mod2)[c(62, 70, 198, 202, 374, 537, 572, 581)]- rstandard(mod2)[c(62, 70, 198, 202, 374, 537, 572, 581)]
##         62         70        198        202        374        537 
## 0.05457370 0.02257026 0.23663538 0.02943293 0.07009602 0.04625260 
##        572        581 
## 0.05869388 0.02076641

None of these points appear to be very influential, as the studentized residuals appear to be similar to the standardized residuals. The point that appears to be most influential is index 198, as the difference is .24. However, this still does not warrant it being called very influential. In case we missed anything, let’s see if there are any points that had a greater difference in studentized and standardized residuals than index 198.

which.max(rstudent(mod2)-rstandard(mod2))
## 198 
## 198

It appears that 198 has the greatest difference in studentized and standardized residuals, suggesting that it is the most influential point. However, this point is not very influential. As one final check for influential points, we’ll return to the plot showing Cook’s distance to see if it shows any influential points.

plot(mod2, 5)

None of the points are outside of a Cook’s Distance of 1, or even 0.5, suggesting that no points are substantially influential.

Because no points are substantially influential and all four methods of determining predictors gave us our current predictors, we will not be making any changes to our data or model in Part 2.

Part 3

Transformations

The following transformations were tested, with mod3 proving to be the best transformation. mod3 transforms the response with a log function. We decided to try a logarithmic transformation to try to help with the constant variance and curvature/ lack of linearity issue we had originally.

mod3= lm(log(Price) ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
    BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
    Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
plot(mod3)

mod4= lm((Price)^2 ~ I(GroundSF^2) + I(YearBuilt^2) + I(BasementSF^2) + I(YearRemodel^2) + I(BasementFinSF^2) + I(GarageSF^2) + I(LotArea^2) + I(Fireplaces^2) + I(LotFrontage^2) + I(Bedroom^2) + I(ScreenPorchSF^2) + I(EnclosedPorchSF^2) + I(HalfBath^2), data = AmesTrain2)
plot(mod4)

mod5= lm((Price)^(1/2) ~ I(GroundSF^(1/2)) + I(YearBuilt^(1/2)) + I(BasementSF^(1/2)) + I(YearRemodel^(1/2)) + I(BasementFinSF^(1/2)) + I(GarageSF^(1/2)) + I(LotArea^(1/2)) + I(Fireplaces^2) + I(LotFrontage^2) + I(Bedroom^2) + I(ScreenPorchSF^2) + I(EnclosedPorchSF^(1/2)) + I(HalfBath^(1/2)), data = AmesTrain2)
plot(mod5)

AmesTrain2$PorchSF = AmesTrain2$ScreenPorchSF + AmesTrain2$EnclosedPorchSF

mod6=lm(Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
    BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
    Bedroom + PorchSF + HalfBath, data = AmesTrain2)
plot(mod6)

mod7=lm(sqrt(Price)~ GroundSF + YearBuilt + BasementSF + YearRemodel + 
    BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage + 
    Bedroom + PorchSF + HalfBath, data = AmesTrain2)
plot(mod7)

As you can see, mod3 performed the best in terms of meeting the conditions of a simple linear model. We’ll discuss this in further detail below.

Part 4

Residual Analysis for Fancier Model

plot(mod3)

hist(mod3$residuals)

Looking at the residuals vs fitted values plot, the linearity condition seems to be met. There appears to be little to no curvature of the residuals, and the residuals hover around zero, suggesting that the line seems to fit the data and no other nonlinear pattern is clear

The zero mean condition seems to be met as well, as the residuals are spread around the zero line in the residuals vs. fitted value plot. The horizontal zero line in the residuals vs. fitted plot seems to pass through the center of the data.

The constant variance condition also appears to be met, as there appears to be equal variance of residuals at all fitted values in the residuals vs. fitted values plot.

Looking at the normal q-q plot and the histogram, the independence condition appears to be met. In the normal q-q plot, the points generally follow the line. The two ends of plot trail off a little, but not by a large amount. The histogram has a general bell curve and does not have a strong skew (it may appear skewed left at first but it is really centered at 0 upon further investigation), further suggesting that the independence condition is met.

Our residual analysis supports that our mod3 fits the conditions of a simple linear model.

Part 5

As our residual analysis suggested that our model fits the conditions of a simple linear model, we made no changes to our model.

newx= data.frame(HouseStyle="2Story", TotalRooms = 9, YearBuilt= 1995, YearRemodel = 2003, LotArea= 11060, LotConfig = "Corner", LotFrontage= 90, Quality = 7, Condition =  5, ExteriorQ = "Gd",  ExteriorC = "Gd", Foundation = "PConc", BasementFinSF=0, BasementSF= 1150, BasementHt = "Ex", BasementFin = "Unf", BasementFBath = 0, BasementHBath= 0, Heating =  "GasA", HeatingQC= "Ex", CentralAir = "Y", GroundSF = 2314, FirstSF= 1164, SecondSF= 1150, Bedroom = 3, FullBath =2, HalfBath = 1, Fireplaces = 1, GarageCars=2, GarageSF= 502, GarageType= "BuiltIn", GarageQ= "TA" , GarageC= "TA", OpenPorchSF= 274, ScreenPorchSF= 0, EnclosedPorchSF=0 )
predict.lm(mod3, newx, interval="prediction", level=.95)
##        fit      lwr      upr
## 1 5.547478 5.211403 5.883553
exp(5.211403)
## [1] 183.3511
exp(5.883553)
## [1] 359.0828

The 95% Prediction Interval for the mean price in thousands at with the conditions listed is (183.3511, 359.0828). This means that we can be 95% confident that the price of an individual house that meets the listed criteria is between 183,351.10 and 359,082.80 dollars.